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Abstract. Most real-world Planning problems are multi-objective, try- 
ing to minimize both the makespan of the solution plan, and some cost 
of the actions involved in the plan. But most, if not all existing ap- 
proaches are based on single-objective planners, and use an aggregation 
of the objectives to remain in the single-objective context. Divide-and- 
Evolve is an evolutionary planner that won the temporal deterministic 
satisficing track at the last International Planning Competitions (IPC). 
Like all Evolutionary Algorithms (EA), it can easily be turned into a 
Pareto-based Multi-Objective EA. It is however important to validate 
the resulting algorithm by comparing it with the aggregation approach: 
this is the goal of this paper. The comparative experiments on a re- 
cently proposed benchmark set that are reported here demonstrate the 
usefulness of going Pareto-based in AI Planning. 



1 Introduction 

Most, if not all, classical AI planning solvers are single-objective. Given a plan- 
ning domain (a set of predicates that describe the state of the system, and a set 
of actions with their pre-requisites and effects), and an instance of this domain 
(a set of objects on which the predicates are instantiated into boolean atoms, 
an initial state and a goal state), classical planners try to find, among the set 
of all feasible plans (sequences of actions such that, when applied to the initial 
state, the goal state becomes true), the one with the minimal number of actions 
(STRIP planning), or with the smallest cost (actions with costs) or with the 
smallest makespan (temporal planning, where actions have durations and can 
be applied in parallel) . A detailed introduction to (single-objective) AI planning 
can be found in [1J. It is clear, however, that most planning problems are in 
fact multi-objective, as the optimal solution in real- world problems often involve 
some trade-off between makespan and cost [3j . A few trials have been made to 
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turn some classical planners into multi-objective optimizers, either using some 
twist in PDDL 2.(0 to account for both makespan and cost [31415] . or using the 
new hooks for several objectives offered by PDDL 3.0 [BJ. However, all these ap- 
proaches are based on a linear aggregation of the different objectives, and were 
not pursued, as witnessed by the new "net-benefit" IPC track, dedicated to ag- 
gregated multiple objectives, that took place in 2006 [7J and 2008 [5], . . . but was 
canceled in 2011 due to a lack of entries. 

In the framework of Evolutionary Algorithms (EAs), Pareto multi-objective 
optimization has received a lot of attention [5], and any single-objective EA can 
"easily" be turned into a multi-objective EA, by modifying the selection step 
(and possible adding some archiving mechanism). Unfortunately, there exist very 
few evolutionary AI planners. Directly evolving plans, as in [10] . obviously does 
not scale up, and was never extended to multi-objective setting. Hence, as far as 
we are aware of, the state-of-the-art in evolutionary AI planning is the previous 
work of some of the authors, Divide-and-Evolve (DaE). DaE evolves variables 
length sequences of states, that start with the problem initial state and end at the 
problem goal state. DaE relies on a classical embedded planner to sequentially 
reach each state of the sequence from the previous one. The concatenation of all 
plans given by the embedded planner is a solution plan of the original problem. 
DaE can thus solve all types of planning problems that the embedded planner 
can solve. Proof-of-concept for DaE was obtained with DaEcpt [H], where the 
embedded planner was CPT, an exact planner |12j - and already included some 
small multi-objective experiments. Since then, the DaE paradigm has evolved, 
and YAHSP a sub-optimal lookahead strategy planning system [T3] is now used 
as the embedded planner [TJ] , and DaEyahsp has reached state-of-the-art results 
in all planning domains 115) , winning the temporal deterministic satisficing track 
at the last IPC in 201 1Q 

The very preliminary work in [llj regarding multi-objective optimization has 
also been recently revisited with DaEyahsp- The lack of existing benchmark 
suite for multi-objective planning led us to extend the small toy problem from 
|llj into a tunable benchmark domain, on which different multi-objectivization 
of DaEyahsp (MO-DaEyahsp) were compared [15] . But because the only other 
approach in AI Planning is the aggregation of the objectives, there is a need to 
compare the multi-objective approach for DaEyahsp with the single-objective 
approach based on the linear aggregation of the objectives: this is the purpose of 
the present work. Section[2]will briefly present planning problems and DaEyahsp 
in the single-objective setting. In Section [3] the multi-objective context will 
be introduced. The multi-objective benchmark suite will be presented, and the 
multi-objectivization of DaEyahsp will be detailed: because YAHSP is a single- 
objective planned, but can be asked to optimize either the makespan or the 
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Fig. 1: Schematic view, and 3 instances, of simple MultiZeno benchmark. 



cost, specific strategies had to be designed regarding how it is called within MO- 
DaEyahsp- Section 2] describes the experimental settings, detailing in particular 
the implementation of the aggregation approach for DaEyahsp an d the intensive 
parameter tuning that was performed for all competing algorithms using the off- 
line problem-independent tuner ParamILS [T7]. The results will be presented 
and discussed in Section [5j and as usual, conclusion and hints about on-going 
and further work will be given in Section [6] 



2 Single-Objective Background 

AI Planning Problems: A planning domain D is defined by a set of object types, 
a set of predicates, and a set of possible actions. An instance is defined by a set 
of objects of the domain types, an initial state, and a goal state. A predicate that 
is instantiated with objects is called an atom, and takes a boolean value. For a 
given instance, a state is defined by assigning values to all possible atoms. An 
action is defined by a set of pre-conditions (atoms) and a set of effects (changing 
some atom values) : the action can be executed only if all pre-conditions are true 
in the current state, and after an action has been executed, the state is modified: 
the system enters a new state. The goal is to find a plan (sequence of actions) 
such that it leads from the initial state to the goal state, and minimizes either 
the number or costs of actions, or the makespan in the case of temporal planning 
where actions have durations and can be run in parallel. 

A simple temporal planning problem in the domain of logistics (inspired 
by the well-known Zeno problem of IPC series) is given in Figure [TJ and will 
be the basis of the benchmark used in this work: the problem involves cities, 
passengers, and planes (object types). Passengers can be transported from one 
city to another (action fly), following the links on the figure. One plane can only 
carry one passenger at a time from one city to another, and the flight duration 



(number on the link) is the same whether or not the plane carries a passenger 
(this defines the domain of the problem). In the simplest non-trivial instance 
of such domain, there are 3 passengers and 2 planes. In the initial state, all 
passengers and planes are in city 0, and in the goal state, all passengers must 
be in city 4. In the default case labeled "Lin." in the table right (forget about 
the costs for now), the not-so-obvious makespan-optimal solution has a total 
makespan of 8 and is left as a teaser for the reader. 

Divide- and- Evolve: Let Vd{I,G) denote the planning problem defined on do- 
main D with initial state / and goal state G. In order to solve Vd(I,G), the 
basic idea of DaEx is to find a sequence of states Si, ■ ■ ■ , S n , and to use some 
embedded planner X to solve the series of planning problems VoiSk, Sf.+i), for 
fc G [0, n] (with the convention that So — I and SVi+i = G). The generation and 
optimization of the sequence of states (<Si)j£ri, n ] is driven by an evolutionary 
algorithm. The fitness of a sequence is computed using the embedded planner 
X, that is called in turn on each of the sub-problems VoiSk, Sfc+i)- The con- 
catenation of the corresponding plans (possibly compressed to take into account 
possible parallelism in the case of temporal planning) is a solution of the initial 
problem. In case one sub-problem cannot be solved by the embedded solver, the 
individual is said unfeasible and its fitness is highly penalized in order to ensure 
that unfeasible individuals are always selected after feasible ones. A thorough 
description of DaEx can be found in [15]. The rest of this section will briefly 
recall the evolutionary parts of DaEx- 

An individual in DaEx is a variable-length list of partial states of the given 
domain (similar to the goal state), and a partial state is a variable- length list 
of atoms (instantiated predicates). The initialization procedure is based on a 
heuristic estimation, for each atom, of the earliest time from which it can become 
true [18] . Furthermore, most existing planners (and this is true for CPT and 
YAHSP, that have been used within DaE) start by computing some partial 
mutual exclusion between possible atoms: this information is also used to reduce 
the search space in DaEx, whenever possible. An individual in DaEx is hence a 
variable-length time-consistent sequence of partial states, and each partial state 
is a variable-length list of atoms that are not pairwise mutually exclusive. 

Crossover and mutation operators are applied with respective user-defined 
probabilities Pcross an d PMut ■ They are defined on the DaEx representation in 
a straightforward manner - though constrained by the heuristic chronology and 
the partial mutex relation between atoms. One-point crossover is adapted to 
variable-length representation: both crossover points are independently chosen, 
uniformly in both parents. Only one offspring is kept, the one that respects the 
approximate chronological constraint on the successive states. Four different 
mutation operators are included, and operate either at the individual level, 
by adding (addState) or removing (delState) a state, or at the state level by 
adding or modifying (addChangeAtom) or removing (delAtom) some atoms in a 
uniformly chose state. The choice among these operators is made according to 
user-defined relative weights (named w-mutationname - see Table Q]). 



3 Multi-Objective Background 



3.1 Pareto-based Multi-Objective Divide-and-Evolve 

Two modifications of DaEyahsp are needed to turn it into an EMOA: use some 
multi-objective selection engine in lieu of the single-objective tournament se- 
lection that is used in the single-objective context; and compute the value of 
both objectives (makespan and cost) for both individuals. The former modifica- 
tion is straightforward, and several alternatives have been experimented within 
[To] . The conclusion is that the indicator-based selection using the hypervolume 
difference indicator [19] performs best - and only this one will be used in the fol- 
lowing, denoted here MO-DaEyahsp- As explained above, the computation of 
the fitness is done by YAHSP- and YAHSP, like all known planners to-date, is 
a single-objective planner. It is nevertheless possible, since PDDL 3.0 [6 , to spec- 
ify other quantities of interest that are to be computed throughout the execution 
of the final plan, without interfering with the search. Within MO-DaEyahsp, 
two strategies are then possible for YAHSP: it can be asked to optimize either 
the makespan or the cost, and to simply compute the cost or the makespan when 
executing the solution plan (for feasible individuals). 

The choice between both strategies is governed by user-defined weights, 
named respectively W- makespan and W-cost (see table [TJ . For each individual, 
the actual strategy is randomly chosen according to those weights, and applied 
to all subproblems of the individual. Note that those weights are tuned using 
ParamlLS (see Section H]), and it turned out that the optimal values for MO- 
DaEyahsp have always been equal weights: something that was to be expected, 
as no objective should be preferred to the other. 

3.2 Aggregation-based Multi-Objective Divide-and-Evolve 

Aggregation is certainly the easiest and most common way to handle multi- 
objective problems with a single-objective optimization algorithm: a series of 
single-objective optimization problems are tackled in turn, the fitness of each 
of these problems is defined by a linear combination of the objectives. In the 
case of makespan and cost, both to be minimized, each linear combination can 
be defined by a single parameter a in [0,1]. In the following, F a will denote 
a * makespan + (1 — a) * cost, and DaEyahsp run optimizing F a will be called 
the a-run. One "run" of the aggregation method thus amounts to running several 
a-runs, and returns the set of non-dominated individuals among the union of all 
final population^- Note that different alpha-runs might have different optimal 
values for their parameters: a complete parameter tuning run of ParamILS must 
be performed for each a-run to ensure a fair comparison with other well-tuned 
approaches. 

The choice of the number of values to choose for the different a depends on 
the available resources. But the choice of the actual values aims at exploring 

7 Some adaptive method has been proposed [20] . where parameter a is adapted on-line, 
spanning all values within a single run: this is left for further work. 
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Fig. 2: Pareto Fronts for the MuLTiZEN06c st problems described in Figured] 

the objective space as uniformly as possible, and some issues might arise if both 
objectives are not scaled similarly. We hence propose here to use some evenly 
spaced values for a (see Section 0} , but only after both objectives have been 
scaled into [0,1]- However, for such scaling to be possible, some bounds must 
be known for each objective. When they are not known, these bounds can be 
approximated from single-objective runs on each of the objectives in turn. 

3.3 Multi-Objective Benchmarks: 

The reader will have by now solved the little puzzle set in Section^ and found the 
solution with makespan 8, that manages to leave no plane idle (detailed solution 
in |16j). In order to turn this problem into a multi-objective one, costs (or risks) 
are added to the fly actions that land in one of the central cities, leading to two 
types of problem: In MuLTiZENOcost, the second objective is the total costs, that 
is accumulated every time a plane lands in a central city; In MuLTiZENO^sfc, the 
second objective is the maximal risk encountered during the complete execution 
of a plan; both are to be minimized. The complexity of the instances can be 
increased by adding more passengers: instances with 3, 6 and 9 passengers will be 
used here. Finally, by tuning the values of the flight durations and the costs/risks, 
different shapes of the Pareto front can be obtained: Figure Q] summarizes three 
possible instances for the MultiZeno domain, and the corresponding Pareto 
fronts for the 6-passengers case are displayed in Figure [2] 

4 Experimental Settings 

Parameter Tuning: It is now widely acknowledged that the large number of pa- 
rameters of most EAs, even though it is a source of flexibility, is also a weakness, 
in that a poor parameter setting can ruin the performances of the most promis- 
ing algorithm. Whereas no generic approach exists for on-line control, there are 
today many available methods for off-line parameter tuning that should be used 
within any evolutionary experiment, in spite of their huge computational cost. 

In this work, unless otherwise stated, the user-defined parameters of both 
MO-DaEyahsp and DaEvahsp shown in Table[T]have been tuned anew for each 
instance, using the ParamILS framework [17]. ParamILS performs an Iterated 



Parameters 


Range 


Description 


W-makcspan 
W-cost 


[0,5] 
[0,5] 


Weight for makespan strategy for YAHSP 
Weight for cost/risk strategy for YAHSP 


Pop-size 


[10,300] 


Population size 


Proba-cross 
Proba-mut 


[0,1] 
[0,1] 


Probability to apply cross-over 
Probability to apply one of the mutation 


w-addatom 
w-addgoal 
w-dclatom 
w-dclgoal 


[1,10] 
[1,10] 
[1,10] 
[1,10] 


Weight for addChangcAtom mutation 
Weight for addGoal mutation 
Weight for dclAtom mutation 
Weight for dclGoal mutation 


Proba-changc 

Proba-dclatom 

Radius 


[0,1] 
[0,1] 
[1,10] 


Probability to change each atom in the addChangcAtom mutation 
Probability to delete each atom in the dclAtom mutation 
Number of neighbour goals to consider for the addGoal mutation 



Table 1: Set of parameters off-line tuned using ParamILS. 



Local Search in the space of possible parameter configurations, evaluating each 
configuration by running the algorithm to be optimized with this configuration 
on the given instance. 

Stopping Criteria: Due to the variable number of calls to YAHSP the number of 
function evaluation is not representative of the CPU effort of runs of DaEyahsp- 
Hence the stopping criterion of all DaEyahsp run was set to a given wall-clock 
time (300, 600 and 1800 seconds for MultiZeno3, 6 and 9 respectively (on an 
Intel(R) Xeon(R) @ 2.67GHz or equivalent). That of MO-DAE Y AHSPwas set 
accordingly: for the sake of a fair comparison, because one run of the aggre- 
gated approach requires n runs of the single-objective version of DaEyahsp ; 
MO-DAEYAHSP w as run for n times the time of each of the DaEyahsp runs. In 
the following, n will vary from 3 to 8 (see Section [5]). The stopping criterion 
for ParamILS was likewise set to a fixed wall-clock time: 48h (resp. 72h) for 
MultiZeno3 and 6 (resp. MultiZeno9), corresponding to 576, 288, and 144 
parameter configuration evaluations for MultiZeno3, 6 and 9 respectively. 

Performance Metrics and Results Visualization: The quality measure used by 
ParamILS to optimize the parameters of both MO-DaEyahsp and each of 
the a-runs of DaEyahsp is the unary hypervolume Ijj- [19] of the set of non- 
dominated points output by the algorithm with respect to the complete true 
Pareto front (only instances where the true Pareto front is fully known have 
been experimented with). The lower the better (a value of indicates that the 
exact Pareto front has been reached). All reported differences in hypervolume 
have been tested using Wilcoxon signed rank test at 95% confidence level, unless 
otherwise stated. 

However, and because the true front is made of a few scattered points (at 
most 17 for MultiZeno9 in this paper), it is also possible to visually monitor the 
empirical Cumulative Distribution Function of the probability to discover each 
point, as well as the whole front. This allows some deeper comparison between 
algorithms even when none has found the whole front. Such hitting plots will be 
used in the following, together with more classical plots of hypervolume vs time. 
Finally, because hitting plots only tell if a given point was reached and do not 



provide any information regarding how far from the other points the different 
runs ended, more details on the approximated Pareto fronts will be given by 
visualizing the merged final populations of all runs of given settings. 

Implementation: For all experiments, 11 independent runs have been performed, 
implemented within the ParadisEO-MOEO frameworl{f|. All performance as- 
sessment procedures (hypervolume calculations, statistical tests), have been achieved 
using the PISA performance assessment too0. 

5 Experimental Results 

This section will compare the Pareto-based MO-DaEyahsp and the aggregation 
approach Agg-DaEyahsp on MultiZeno3, 6 and 9. Unless otherwise stated, 
the default domain definition leading to a linear Pareto front (see Figure [1] and 
[5]-left) will be used, and one Agg-DaEyahsp run will be made of 7 different 
a-runs, with a taking the values 0,0.1,0.3,0.5,0.7,0.9, and 1. 

The MultiZenoS Problem proved to be too easy: both MO-DaEyahsp and Agg- 
DaEyahsp nn d the complete Pareto fronts, and the hitting plots reach 100% 
in less than 80s (resp. 90s) for the Cost (resp. Risk) version of the instance 
(not shown here). MO-DaEyahsp is slightly slower (resp. faster) than Agg- 
DaEyahsp in the Cost (resp. Risk) instance, but no significant difference is to 
be reported. Only instances -6 and -9 will be looked at in the following. 

The Risk Objective: On these instances, however, the Risk objective proved 
to be almost too difficult to be of interest here, even though there are only 
3 points on the Pareto Front, whatever the number of passengers: as can be 
seen on Figure O no algorithm could identify the complete Pareto front for the 
MultiZeno9 instance (line 4); for MultiZeno6 (line 2), MO-DaEyahsp could 
reliably identify the whole front (in 9 runs out of 11), while only a single run 
of Agg-DaEyahsp could identify the middle point (40,20). M O-DaEyahsp is 
hence a clear winner here - however, too little information is brought by the 
risk value, as one single stop in a risky station will completely hide the possibly 
low-risk remaining of the plan. Further work will aim at designing a smoother 
fitness for such situations. 

The rest of the paper will hence concentrate on the Cost versions of Multi- 
Zeno6 and 9 (simply denoted MultiZeno{6,9}), where significant differences 
between both approaches can be highlighted. 

Results on the Default Instance: From the plots of the evolution of the average 
hypervolumes (Figure [3]), MO-DaEyahsp is the winner for MultiZeno6, and 
Agg-DaEyahsp is the winner for MultiZeno9. Taking a closer look at the 
hitting plots (Figure [5]), we can see for MultiZeno6 (line 1) that all runs of 




Fig. 3: Evolution of hypervolume for DaEyahsp (green squares) and Agg- 
DaEyahsp (blue triangles) for MultiZeno6 (left) and MultiZeno9 (right). 
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Fig. 4: Pareto fronts approximations (union of all final populations). 



MO-DaEyahsp reach the complete Pareto front in around 2500s, while only 9 
runs out of 11 do reach it. On the other hand, for MultiZeno9, and though 
the figures of line 3 are more difficult to read because they contain the CDF 
for 17 points, slightly more points seem to be reached by Agg-DaEyahsp than 
by MO-DaEyahsp- Looking now at the approximations of the Pareto fronts 
(Figure|l|), the fronts returned by Agg-DaEyahsp for MultiZeno6 show a large 
dispersion away from the true front, whereas the same figure for MO-DaEyahsp 
(not shown) only contains the true front. Regarding MultiZeno9, even though 
it reaches less points from the true front, MO-DaEyahsp demonstrates a much 
more robust behavior than Agg-DaEyahsPj for which the approximate fronts 
are, again, quite dispersed, sometimes far from the true front. 



Results on other MultiZeno6 instances: Further experiments have been con- 
ducted on different variants of MultiZeno6 instance, described in Figure [TJ 
The corresponding hitting plots can be seen on Figure [5] As in the Linear de- 
fault case, MO-DaEyahsp is a clear winner - and this is confirmed by the plots 
of the approximate Pareto fronts (not shown), for which Agg-DaEyahsp again 
shows a much larger dispersion away from the true front than MO-DaEyahsp- 
All results presented until now have been obtained by first optimizing the pa- 
rameters of all algorithms with ParamILS. Interestingly, when using the param- 
eters optimized by ParamILS for the Linear instance on these other instances, 
the results are only slightly worse: this observation will motivate further work 
dedicated to the generalization of the parameter tuning across instances. 



Fig. 5: Hitting plots for MO-DaE Y ahsp (left) and Agg-DaE Y ahsp (right), for 
instances 2, 3, and 4 of MultiZeno6 from Figure [T] (from top to bottom). 

6 Discussion and Conclusion 

The experiments presented in this paper have somehow demonstrated the greater 
efficiency of the Pareto-based approach to multi-objective AI Planning MO- 
DaEyahsp compared to the more traditional approach by aggregation of the 
objectives Agg-DaEyahsp- The case is clear on MultiZeno6, and on the dif- 
ferent instances that have been experimented with, where MO-DaEyahsp ro- 
bustly finds the whole Pareto front (except for the Convex instance), whereas 
Agg-DaEyahsp performs much worse in all aspects. This is also true on the 
MultiZeno9 instance, in spite of the better hypervolume indicator: indeed, a 
few more points on the Pareto front are found a little more often, but the global 
picture remains a poor approximation of the Pareto front. Other experiments 
on more instances are needed to confirm these first results, and on-going work 
is concerned with solving instances generated from IPC benchmarks by merging 
the cost and the temporal domains when the same instances exist in both. 

Regarding the computational cost, one Agg-DaEyahsp ru n requires several 
single-objective runs - and as many parameter tuning procedures. We have cho- 
sen here to use 7 different values for a, and it was clear from results not shown 
here that taking away a few of these resulted in a decrease of quality of the 
results. The computational cost of the parameter tuning could be reduced, too: 
first, a complete tuning anew for each instance is unrealistic, and was only done 
here for the sake of a fair comparison between both approaches; second, even on 
a single instance, it should be possible to tune all parameters (except those of 
YAHSP strategy) for all a-runs together. Finally, one of the most promising di- 
rections for future research is the on-line tuning of YAHSP strategy, e.g., using 
a self-adaptive approach, where the strategies are attached to the individual. 



Fig. 6: Hitting plots for MO-DaE Y ahsp (left) and Agg-DaE Y ahsp (right), 
for instances (from top to bottom) MultiZeno6cos*, MuLTiZEN06_Ri S fc, 
MuLTiZEN09cost, and MuLTiZEN09_R is fc. The lower line on each plots is the 
experimental CDF for the probability to reach the whole Pareto front. 
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Preface 



This textbook is intended for use by students of physics, physical chemistry, 
and theoretical chemistry. The reader is presumed to have a basic knowledge 
of atomic and quantum physics at the level provided, for example, by the first 
few chapters in our book The Physics of Atoms and Quanta. The student of 
physics will find here material which should be included in the basic education 
of every physicist. This book should furthermore allow students to acquire an 
appreciation of the breadth and variety within the field of molecular physics and 
its future as a fascinating area of research. 

For the student of chemistry, the concepts introduced in this book will provide 
a theoretical framework for that entire field of study. With the help of these con- 
cepts, it is at least in principle possible to reduce the enormous body of empirical 
chemical knowledge to a few basic principles: those of quantum mechanics. In 
addition, modern physical methods whose fundamentals are introduced here are 
becoming increasingly important in chemistry and now represent indispensable 
tools for the chemist. As examples, we might mention the structural analysis of 
complex organic compounds, spectroscopic investigation of very rapid reaction 
processes or, as a practical application, the remote detection of pollutants in the 
air. 
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1 Fixed-Period Problems: The Sublinear Case 

With this chapter, the preliminaries are over, and we begin the search for periodic 
solutions to Hamiltonian systems. All this will be done in the convex case; that 
is, we shall study the boundary-value problem 

x = JH'(t,x) 
x(0) = x{T) 

with H(t, •) a convex function of x, going to +00 when ||x|| — > 00. 
1.1 Autonomous Systems 

In this section, we will consider the case when the Hamiltonian H(x) is au- 
tonomous. For the sake of simplicity, we shall also assume that it is C x . 

We shall first consider the question of nontriviality, within the general frame- 
work of (Aoo, _Boo)-subquadratic Hamiltonians. In the second subsection, we shall 
look into the special case when H is (0, 6oo)-subquadratic, and we shall try to 
derive additional information. 

The General Case: Nontriviality. We assume that H is ( A 00, Boo)- sub- 
quadratic at infinity, for some constant symmetric matrices Aoo and B^, with 
Boo — Aoc positive definite. Set: 



7 : — smallest eigenvalue of B^ — A,. 



oc 



(1) 



A : = largest negative eigenvalue of J— + A, 



dt 



OO 



(2) 



Theorem [T] tells us that if A + 7 < 0, the boundary- value problem: 



x = JH'{x) 
x{0) = x{T) 



(3) 



has at least one solution x, which is found by minimizing the dual action func- 
tional: 

" "1 



ip(u) 



(A-^u) +N*{-u) 



dt 



(4) 



on the range of A, which is a subspace R{A) 2 L with finite codimension. Here 



N{x) := H{x) - - {A^x.x) 



is a convex function, and 



1 



N(x) < - {{Boo -A ao )x,x) +c \Jx 

Proposition 1. Assume H'(0) = and H(0) = 0. Set: 

5 := liminf 2N{x) lldl" 2 . 

x— >0 

If 7 < —A < 5, the solution u is non-zero: 

x{t) ^0 Vi . 



(5) 



(6) 



(7) 



(8) 



Proof. Condition ([7J means that, for every 5' > 5, there is some e > such that 

(9) 



\x\\ < e =► N{x) < 5 - \\x\\ 2 



It is an exercise in convex analysis, into which we shall not go, to show that 
this implies that there is an r\ > such that 



f\\x\\<V^N*{y)<^\\yf 



(10) 



Fig. 1. This is the caption of the figure displaying a white eagle and a white horse on 
a snow field 
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Since u\ is a smooth function, we will have H/iuiH^ < rj for h small enough, 
and inequality (|10[) will hold, yielding thereby: 



i>(hui) < \\uif 2 + ||ui|| 2 



(11) 



If we choose 6' close enough to S, the quantity (-^ + will be negative, and 
we end up with 

i>(hui) < for h ^ small . (12) 

On the other hand, we check directly that V'(O) = 0. This shows that cannot 
be a minimizer of ip, not even a local one. So u ^ and u ^ A~ 1 (0) = 0. □ 



2 and (aoo^b^-subquadratic at infinity. Let £i, 
, £jv &e i/ie equilibria, that is, the solutions of H'(^) = 0. Denote by ojk the 



Corollary 1. Assume H is C and (000,600 



smallest eigenvalue of H" (£&), and set: 

u! := Min {u>i, . . . , w^} 



T 

— 600 < —E 

Z7T 



a~ 

2tt ° 



T 
2?r 



(13) 
(14) 



i/ien minimization of i\) yields a non-constant T-periodic solution x. 



We recall once more that by the integer part E[a] of a e H, we mean the 
a G 2Zi such that a < a < a + 1. For instance, if we take doo = 0, Corollary 2 
tells us that x exists and is non-constant provided that: 



T , T 
— 60c < 1 < — 
2tt 2tt 



or 



2tt 2tt 

W ' 600 



(15) 
(16) 



Proof. The spectrum of A is l^r-Z? 
given by ^k + Ooo, where 



rp ft O 



27T,, 

aoo < < — (k 



Ooo- The largest negative eigenvalue A is 
l) + aoo- (17) 



Hence: 



k a = E 



T 



The condition 7 < — A < 5 now becomes: 

27T 



< 



T 



k a - Ooo < u ■ 



which is precisely condition (fT4 



(18) 

(19) 
□ 
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Lemma 1. Assume that H is C 2 on R 2 "\{0} and that H"(x) is non-degenerate 
for any x ^= 0. Then any local minimizer x of ' i\) has minimal period T. 

Proof. We know that x, or x + £ for some constant £ £ H 2 ™, is a T-periodic 
solution of the Hamiltonian system: 

x = JH'{x) . (20) 

There is no loss of generality in taking £ = 0. So ip(x) > ip(x) for all x in 
some neighbourhood of x in IT 1 ' 2 (TR/TZ; TR 2n ) . 

But this index is precisely the index it(x) of the T-periodic solution x over 
the interval (0,T), as defined in Sect. 2.6. So 

i T {x) = . (21) 

Now if x has a lower period, T/k say, we would have, by Corollary 31: 

i T (x) = i k T/k(x) > ki T / k (x) + fc-l>fc-l>l . (22) 

This would contradict ([2T]l . and thus cannot happen. □ 

Notes and Comments. The results in this section are a refined version of pQ; the 
minimality result of Proposition 14 was the first of its kind. 

To understand the nontriviality conditions, such as the one in formula (|16j) . 
one may think of a one-parameter family xt, T £ (27ra; _1 , 27t6^ c 1 ) of periodic 
solutions, £t(0) = xt(T), with xt going away to infinity when T — > 2ituj~ 1 , 
which is the period of the linearized system at 0. 



Table 1. This is the example table taken out of The TftXbook, p. 246 



Year 


World population 


8000 B.C. 


5,000,000 


50 A.D. 


200,000,000 


1650 A.D. 


500,000,000 


1945 A.D. 


2,300,000,000 


1980 A.D. 


4,400,000,000 



Theorem 1 (Ghoussoub-Preiss). Assume H(t,x) is (0, e)-subquadratic at 
infinity for all e > 0, and T-periodic in t 

H(t, •) is convex Vf (23) 

H(-,x) is T-periodic Vx (24) 
H (t, x) > n (\\x\\) with n(s)s _1 — » oo as s — » oo (25) 
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Ve>0, 3c : H(t,x) < ^\\xf + c . (26) 

Assume also that H is C 2 , and H"(t,x) is positive definite everywhere. Then 
there is a sequence Xk, k <E IN, of kT -periodic solutions of the system 

x = JH'(t,x) (27) 

such that, for every k S IN, there is some p a e IN with: 

P > Po => x pk ^ x k . (28) 

□ 

Example 1 (External forcing ). Consider the system: 

x = JH'{x) + f(t) (29) 

where the Hamiltonian H is (0, &oo)-subquadratic, and the forcing term is a 
distribution on the circle: 

f=j t F + f Q with F e L 2 (m,/T2Z; H 2n ) , (30) 
where f := T _1 f(t)dt. For instance, 

/(*) = E « . ( 31 ) 

fceiN 

where 5 k is the Dirac mass at t = k and £ G 1R 2 " is a constant, fits the pre- 
scription. This means that the system x = JH'(x) is being excited by a series 
of identical shocks at interval T. 

Definition 1. Let A^^t) and i?oo(£) be symmetric operators in JR 2n , depending 
continuously on t G [0,T], such that A^^) < -Boo(i) for all t. 

A Borelian function H : [0,T] x H 2 " JR is called (A^, B^-subquadratic 



at infinity if there exists a function N(t, x) such that: 

H(t,x) = ^{A 00 {t)x,x)+N(t,x) (32) 

Vf , N(t, x) is convex with respect to x (33) 

N(t, x) > n{\\x\\) with n(s)s^ 1 -> +oo as s ->• +oo (34) 

BcelR : x) < i (Boo^x,^) +c Vx . (35) 



If A^t) = aool and B^t) = b^I, with < s H, we s/mZZ say i/iai 
-ff is (doo, boo) -subquadratic at infinity. As an example, the function ||x|| a , with 
1 < a < 2, is (0, e)-subquadratic at infinity for every e > 0. Similarly, the 
Hamiltonian 

H{t,x) = \k\\k\\ 2 + \\x\r (36) 
is (fc, fc + e)-subquadratic for every e > 0. iVoie fftaf, if k < 0, it is not convex. 
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Notes and Comments. The first results on subharmonics were obtained by Ra- 
binowitz in [5] , who showed the existence of infinitely many subharmonics both 
in the subquadratic and superquadratic case, with suitable growth conditions 
on H' . Again the duality approach enabled Clarke and Ekeland in 2, to treat 
the same problem in the convex-subquadratic case, with growth conditions on 
H only. 

Recently, Michalek and Tarantello (see [3] and [4]) have obtained lower bound 
on the number of subharmonics of period kT, based on symmetry considerations 
and on pinching estimates, as in Sect. 5.2 of this article. 
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1 Fixed-Period Problems: The Sublinear Case 

With this chapter, the preliminaries are over, and we begin the search for periodic 
solutions to Hamiltonian systems. All this will be done in the convex case; that 
is, we shall study the boundary-value problem 

x = JH'(t,x) 
x(0) = x{T) 

with H(t, •) a convex function of x, going to +00 when ||x|| — > 00. 
1.1 Autonomous Systems 

In this section, we will consider the case when the Hamiltonian H(x) is au- 
tonomous. For the sake of simplicity, we shall also assume that it is C . 

We shall first consider the question of nontriviality, within the general frame- 
work of (Aoo, _Boo)-subquadratic Hamiltonians. In the second subsection, we shall 
look into the special case when H is (0, 6 OQ )-subquadratic, and we shall try to 
derive additional information. 

The General Case: Nontriviality. We assume that H is (^4oo, -Boo)- sub- 
quadratic at infinity, for some constant symmetric matrices Aoo and B^, with 
Boo — ^4oo positive definite. Set: 

7 : = smallest eigenvalue of B^ — Aoo (1) 
A : = largest negative eigenvalue of J^- + . (2) 



Theorem 21 tells us that if A + 7 < 0, the boundary-value problem: 



x = JH'(x) 
x(0) = x{T) 



(3) 



has at least one solution x, which is found by minimizing the dual action func- 
tional: 

" "1 



ip(u) 



(A-y u) +N*{-u) 



dt 



(4) 



on the range of A, which is a subspace R{A) 2 L with finite codimension. Here 



N{x) := H{x) - - {A^x.x) 



is a convex function, and 



1 



N{x) -^-{{Boo- A OQ )x,x)+c \Jx 

Proposition 1. Assume H'(0) = and H(0) = 0. Set: 

5 := liminf 2N(x) Ibll" 2 . 

x— >0 

If 7 < —A < 5, the solution u is non-zero: 

x{t) ^0 Vi . 



(5) 



(6) 



(7) 



(8) 



Proof. Condition ([7J means that, for every 5' > 5, there is some e > such that 

(9) 



\x\\ < e =► N(x) < 5 - \\x\\ 2 



It is an exercise in convex analysis, into which we shall not go, to show that 
this implies that there is an r\ > such that 



f\\x\\<V^N*(y)<^\\yf 



(10) 



Fig. 1. This is the caption of the figure displaying a white eagle and a white horse on 
a snow field 
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Since u\ is a smooth function, we will have H/iuiH^ < rj for h small enough, 
and inequality (|10[) will hold, yielding thereby: 



i>(hui) < \\uif 2 + ||ui|| 2 



(11) 



If we choose 6' close enough to S, the quantity (-^ + will be negative, and 
we end up with 

i>(hui) < for h ^ small . (12) 

On the other hand, we check directly that V'(O) = 0. This shows that cannot 
be a minimizer of ip, not even a local one. So u ^ and u ^ A~ 1 (0) = 0. □ 



2 and (aoo^b^-subquadratic at infinity. Let £i, 
, £jv &e i/ie equilibria, that is, the solutions of H'(^) = 0. Denote by ojk the 



Corollary 1. Assume H is C and (000,600 



smallest eigenvalue of H" (£&), and set: 

u! := Min {u>i, . . . , w^} 



T 

— 600 < —E 

Z7T 



a~ 

2tt ° 



T 
2?r 



(13) 
(14) 



i/ien minimization of i\) yields a non-constant T-periodic solution x. 



We recall once more that by the integer part E[a] of a e H, we mean the 
a G 2Zi such that a < a < a + 1. For instance, if we take doo = 0, Corollary 2 
tells us that x exists and is non-constant provided that: 



T , T 
— 60c < 1 < — 
2tt 2tt 



or 



2tt 2tt 

W ' 600 



(15) 
(16) 



Proof. The spectrum of A is l^r-Z? 
given by ^k + Ooo, where 



rp ft O 



27T,, 

aoo < < — (k 



Ooo- The largest negative eigenvalue A is 
l) + aoo- (17) 



Hence: 



k a = E 



T 



The condition 7 < — A < 5 now becomes: 

27T 



< 



T 



k a - Ooo < u ■ 



which is precisely condition (fT4 



(18) 

(19) 
□ 
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Lemma 1. Assume that H is C 2 on R 2 "\{0} and that H"(x) is non-degenerate 
for any x ^= 0. Then any local minimizer x of ' i\) has minimal period T. 

Proof. We know that x, or x + £ for some constant £ £ H 2 ™, is a T-periodic 
solution of the Hamiltonian system: 

x = JH'{x) . (20) 

There is no loss of generality in taking £ = 0. So ip(x) > ip(x) for all x in 
some neighbourhood of x in IT 1 ' 2 (TR/TZ; TR 2n ) . 

But this index is precisely the index it(x) of the T-periodic solution x over 
the interval (0,T), as defined in Sect. 2.6. So 

i T {x) = . (21) 

Now if x has a lower period, T/k say, we would have, by Corollary 31: 

i T (x) = i k T/k(x) > ki T / k (x) + fc-l>fc-l>l . (22) 

This would contradict ([2T]l . and thus cannot happen. □ 

Notes and Comments. The results in this section are a refined version of[l980; 
the minimality result of Proposition 14 was the first of its kind. 

To understand the nontriviality conditions, such as the one in formula (|16j) . 
one may think of a one-parameter family xt, T £ (27ra; _1 , 27t6^ c 1 ) of periodic 
solutions, xt(0) = xt(T), with xt going away to infinity when T — > 2ituj~ 1 , 
which is the period of the linearized system at 0. 



Table 1. This is the example table taken out of The TftXbook, p. 246 



Year 


World population 


8000 B.C. 


5,000,000 


50 A.D. 


200,000,000 


1650 A.D. 


500,000,000 


1945 A.D. 


2,300,000,000 


1980 A.D. 


4,400,000,000 



Theorem 1 (Ghoussoub-Preiss). Assume H(t,x) is (0, e)-subquadratic at 
infinity for all e > 0, and T-periodic in t 

H(t, •) is convex Vf (23) 

H(-,x) is T-periodic Vx (24) 
H (t, x) > n (\\x\\) with n(s)s _1 — » oo as s — » oo (25) 
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Ve>0, 3c : H(t,x) < ^\\xf + c . (26) 

Assume also that H is C 2 , and H"(t,x) is positive definite everywhere. Then 
there is a sequence Xk, k <E IN, of kT -periodic solutions of the system 

x = JH'(t,x) (27) 

such that, for every k S IN, there is some p a e IN with: 

P > Po => x pk ^ x k . (28) 

□ 

Example 1 (External forcing ). Consider the system: 

x = JH'{x) + f(t) (29) 

where the Hamiltonian H is (0, &oo)-subquadratic, and the forcing term is a 
distribution on the circle: 

f=j t F + f Q with F e L 2 (m,/T2Z; H 2n ) , (30) 
where f := T _1 f(t)dt. For instance, 

/(*) = E « . ( 31 ) 

fceiN 

where 5 k is the Dirac mass at t = k and £ G 1R 2 " is a constant, fits the pre- 
scription. This means that the system x = JH'(x) is being excited by a series 
of identical shocks at interval T. 

Definition 1. Let A^^t) and i?oo(£) be symmetric operators in JR 2n , depending 
continuously on t G [0,T], such that A^^) < -Boo(i) for all t. 

A Borelian function H : [0,T] x H 2 " JR is called (A^, B^-subquadratic 



at infinity if there exists a function N(t, x) such that: 

H(t,x) = ^{A 00 {t)x,x)+N(t,x) (32) 

Vf , N(t, x) is convex with respect to x (33) 

N(t, x) > n{\\x\\) with n(s)s^ 1 -> +oo as s ->• +oo (34) 

BcelR : x) < i (Boo^x,^) +c Vx . (35) 



If A^t) = aool and B^t) = b^I, with < s H, we s/mZZ say i/iai 
-ff is (doo, boo) -subquadratic at infinity. As an example, the function ||x|| a , with 
1 < a < 2, is (0, e)-subquadratic at infinity for every e > 0. Similarly, the 
Hamiltonian 

H{t,x) = \k\\k\\ 2 + \\x\r (36) 
is (fc, fc + e)-subquadratic for every e > 0. iVoie fftaf, if k < 0, it is not convex. 
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Notes and Comments. The first results on subharmonics were obtained by Rabi- 
nowitz in 1985 , who showed the existence of infinitely many subharmonics both 
in the subquadratic and superquadratic case, with suitable growth conditions 
on H'. Again the duality approach enabled Clarke and Ekeland in 119811 to treat 
the same problem in the convex-subquadratic case, with growth conditions on 
H only. 

Recently, Michalek and Tarantello (see Michalek, R., Tarantello, G. 1982]and 
Tarantcllo, G. I1983|) have obtained lower bound on the number of subharmonics 
of period kT, based on symmetry considerations and on pinching estimates, as 
in Sect. 5.2 of this article. 
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